1. Field of the Invention
The present invention relates to an image processing apparatus and method used to determine whether an image to be processed is obtained by photographing a person as a primary subject, the apparatus and method for detecting the face of a person included in an image read by an image input device such as an imaging device.
2. Description of the Related Art
Imaging devices, particularly, digital still cameras (hereinbelow, referred to as digital cameras), which are being widely spread recently, each include various image processing functions: an exposure adjustment mechanism called auto gain control (AGC) for assigning a subject brightness threshold to the dynamic range of an imaging device on the basis of image signals taken by the imaging device comprising a CCD or a CMOS sensor; a tone adjustment mechanism called auto white balance (AWB) for correcting the tone of a subject to a proper tone; a gamma conversion mechanism for adjusting a change in gray scale and contrast; a noise removal mechanism; and an edge enhancement mechanism.
As described in Japanese Patent Laid-Open Nos. 2000-123164 and 2000-123165, recently, processes for taken image signals are being put into intelligence.
Japanese Patent Laid-Open No. 2000-123164 describes that change of a process or a conversion level in accordance with the features of an image is more preferable than execution of a predetermined process without consideration of the features of the image. Japanese Patent Laid-Open No. 2000-123165 discloses correction with emphasis on a primary subject of an image, namely, image correction appropriate to the primary subject. Particularly, for a person photograph including a person as a primary subject, in a case where a photographed image is displayed on a display screen or is printed as a hard copy output through a color printer to obtain a playback image, when various image enhancement processes or image correction processes such as color reproduction for the skin tone of the person and contrast control are different from those for other images, a person viewing the playback image receives a good impression. This fact is known.
For methods for extracting the face of a person, included in a photographed image, from the image, several methods are disclosed. For example, Japanese Patent Laid-Open No. 2001-309225 discloses a method for detecting a face using a first face candidate detection algorithm and a second algorithm. The first face candidate detection algorithm uses a pattern recognition operator for detecting a central part which may probably include the skin of a subject person on the basis of colors and shapes and detecting surrounding areas which may probably include the hair thereof on the basis of the colors and shapes to detect an area (face candidate area) which may probably include the face thereof. The second algorithm uses pattern matching to detect the presence of the face in the face candidate area obtained by the first algorithm.
Japanese Patent Laid-Open No. 8-63595 discloses the following methods. According to a first method, the edge of a skin tone area is extracted on the basis of colors, and the extracted edge is compared to a prepared face edge template to obtain the degree of matching. On the basis of the degree of matching, a face is detected. According to a second method, an eye candidate area is obtained and is compared to an eye template to obtain the degree of matching. On the basis of the degree of matching, a face is detected. According to a third method, a face candidate area obtained using a face edge template is subjected to two-dimensional Fourier transformation, prepared face template images each including eyes, a nose, a mouth, and hair are subjected to two-dimensional Fourier transformation, the amount of features to be defined is obtained on the basis of the above two-dimensional Fourier transformation results, and the amount of features is subjected to a threshold process, thus detecting a face.
Japanese Patent Application No. 2001-282283 discloses the following method: An eye candidate area is obtained. When a plurality of eye candidates are found, surrounding areas are arranged in the vicinities of an arbitrary pair of eye candidates so as to correspond to the respective eye candidates. On the basis of the directional difference between a predetermined reference gradient and a gradient in pixel value between each pixel in the surrounding area and the neighboring pixels, face detection is performed.
Recent digital cameras each have an angle (inclination) sensor therein. The angle sensor detects the photographing position of the camera (whether the camera is held longitudinally or laterally, or how much the camera is rotated from a reference position) when an image is photographed. The photographing position is set to information attached to the image. The digital camera outputs the image together with the photographing-position information as image attached information. This type of digital camera appears on the market.
General face detection methods including the above-mentioned methods for extracting the face of a person are performed on the precondition that a face to be detected is photographed so that the face of a person in an upright position is fundamentally included in an image frame. Accordingly, the orientation of the face which can be detected is restricted.
However, for photographed images, particularly, images photographed by digital cameras, a photographer holds a camera freely, for example, longitudinally, laterally, or tilts the camera in some cases. In addition, the position of a subject person is not always upright so that the whole face is photographed like a photograph for an ID card. The person may slightly tilt his or her face and he or she may be lying. Various positions can be expected.
Accordingly, a face may be freely photographed at various angles with respect to the image frame, fundamentally, in an angle range of 360 degrees. In order to detect a face at any angle, therefore, it is necessary to repeat a process of detecting a face from an image, obtained by rotating an original image, in predetermined angle increments, the predetermined angle being a reference angle range in which the face detecting process can be performed. Alternatively, it is necessary to prepare templates for pattern matching and various amounts of features used in face detection so as to correspond to all of predetermined angles and execute a face detecting process at each of the predetermined angles. In order to detect a face from a snapshot taken by a digital camera, these methods require detection time proportional to the number of times of the detecting process repeated in predetermined angle increments.